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In statistical mechanics, we consider quantum mechanical 
systems in which particles like atoms and electrons have 
different energy levels. 

We want to know the probability that the particles are at each 
level. We can use this information to calculate the population 
at each level, the total energy of the system, the heat capacity, 
etc. 

Later on, we shall learn how to calculate this probability using 
the Boltzmann distribution. Before that, we need to answer 
one question: 

Even if we know the probabilities at each level, particles are 
often free to hop between levels. Would this not cause big 
changes in energy of the system? 

In order to answer this question, we investigate the example 
where we only have 2 energy levels, and with equal probabilities. 



Basic statistics 



To use a familiar example, think of each particle as a coin and 
each level as one side of the coin. 

Suppose the coin is tossed 2 times. The possible outcomes are: 

TT 

HT 

TH 

HH 

Let x be the number of heads. Let n be the total number of 
outcomes. 

Let P(x) be probability of getting x heads. This is given by 

P(x) = x/n. 

P(0) = 1/4 
P(l) = 2/4 
P(2) = 1/4 
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Mean and standard deviation 



Suppose we toss the coin n times and record the number of H. 
Suppose we repeat this many times. 

We would get a sample of data for x, e.g. {2, 0, 1, 3, 2, 2, 3, 
0, ... } 

In statistics, we are often interested in two numbers - the mean 
and the standard deviation. 

The mean is the simple average. 

The standard deviation tells us how much the numbers in the 
data deviate from the mean. 
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When the standard deviation is small 

The standard deviation is relevant to the idea of macrostates 
that we want to develop. 

We are particularly interested in what it means if the standard 
deviation becomes very small. 

Recall that the standard deviation is a measure of the average 
deviation of x from the mean. 

So if the standard deviation is small, it means that most of the 
data is very close to the mean value. 



Large standard deviation 
P(x) 



Small standard deviation 



P(x) 



If we look at a histogram of the data, it would look more 
sharply peaked. 
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If we know the probaility distribution P(x), we can also 
calculate the mean and standard deviation without actually 
tossing the coin. 

The mean is given by 

The standard deviation is the root mean square of x — /x. So it 
is essentially a kind of average of how much x deviates from fj,. 

It is given by 

ct = \/E^-m) 2 ^W. 



The binomial distribution 



Let p be probability of getting a Head, and q the probaility of 
getting a Tail. 

If we have a fair coin, then p = q = 1/2. 
Suppose we toss a coin N times. 

The probaility of getting x heads is given by the binomial 
distribution: 

p(x) = N C n p n q N - n - 
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Mean and standard deviation 

When we toss a coin N times, suppose that we get x heads. 

When we toss N times again, we may get a different x. 

The mean value of x is given by a sum over x, weighted by the 
probabilities: 

N 

fj, = ^ xP(x) = Np. 

x=0 

The standard deviation is given by: 

N 



^(x- M ) 2 P(x)=0VpQ. 
\ x=0 



Both formulae can be derived by substituting the binomial 
distribution P{x) = N C n p n q N ~ n - 



A reference quantity 

The reason is because, on the graph, we are looking at the 
standard deviation relative to the total number of coins N. 

So, although 10 is large than 1, 10/400 would be smaller than 
1/4. 

In the case of the coins, we should therefore be comparing the 
standard deviation with the total number of coins N. 

x is the actual number of heads. 

The average of deviation of this number from the mean, as a 
fraction of N, is a/N = Jpq/N. 

When N is very large, this fraction approaches zero. 
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Distribution of the number of Heads 



Lets get some idea of how a changes a iV is increased. For a 
fair coin p = q = 1/2. 



N o(= VNpq) 
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It looks like the number of heads changes more from one toss 
to the next, as the number of coins is increased. 

But if we plot the distribution on a graph: 
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we would find that it gets more sharply peaked! 
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The dotted curve is the smallest N, the solid curve the largest 
N. Notice that the peak gets narrower for large N. 

If N is as big as the number of atoms in a real material, the 
peak becomes extremely narrow. 

Then x = N/2 becomes almost the only possible value. 
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The Analogy with Temperature 



We can apply this to atoms: 

1. H is the higher energy state, T is the lower one. 

2. One toss of a coin is one atom. 

3. The larger the number of H, the higher the temperature. 

In a real material at room temperature, atoms could hop 
between energy levels randomly through colliding with other 
atoms. 

Because the number of atoms is huge, deviation of particle 
numbers at each level from the values predicted from their 
probabilities are extremely small. 
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Mole Concept 



Exercise 1 

How many atoms are there in each of the following? 
(N A = 6.022 x 10 23 ) 

1) 1 mole of iron 

2) 0.2 mole of copper 

3) 3 moles of silver 

4) 1 mole of oxygen (O2), and how many molecules? 

5) 1 mole of water (H2O), and how many oxygen atoms? 

6) 1 mole of sodium chloride (NaCI), and how many sodium 
ions? 

7) 1 mole of helium-4 ( 4 He) 

8) 1 mole of helium-3 ( 3 He) 
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Warming up exercises 
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Mole Concept 



Solutions to Exercise 1 

1) 6.022 x 10 23 

2) 0.2 x 6.022 x 10 23 

3) 3 x 6.022 x 10 23 

4) 2 x 6.022 x 10 23 atoms, and 6.022 x 10 23 molecules 

5) 3 x 6.022 x 10 23 atoms, and 6.022 x 10 23 oxygen atoms 

6) 2 x 6.022 x 10 23 atoms, and 6.022 x 10 23 sodium ions 

7) 6.022 x 10 23 

8) 6.022 x 10 23 
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Mole Concept 



Exercise 2 

What is the mass of each of the following? (A r stands for 
relative atomic mass.) 

1) 1 mole of iron (A r =56) 

2) 0.2 mole of copper (A r =64) 

3) 3 moles of silver (A, =108) 

4) 1 mole of oxygen (O2) (A r =16) 

5) 1 mole of water (H 2 0) (A r of H = l, of 0=16) 

6) 1 mole of sodium chloride (NaCI) (A r of Na=23, of 
Cl=35.5) 

7) 1 mole of helium-4 ( 4 He) 

8) 1 mole of helium-3 ( 3 He) 
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Mole Concept 



Exercise 3 

How many atoms are there in each of the following? (A r 
stands for relative atomic mass.) 

1) 1 g of iron (A r =56) 

2) 1 g of copper (A r =64) 

3) 1 g of silver (A r =108) 

4) 1 g of oxygen (0 2 ) (A r =16) 

5) 1 g of water (H 2 0) (A r of H = l, of 0=16) 

6) 1 g of sodium chloride (NaCI) (A r of Na=23, of Cl=35.5) 

7) 1 g of helium-4 ( 4 He) 

8) 1 g of helium-3 ( 3 He) 
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Mole Concept 



Solutions to Exercise 2 

1) 56 g 

2) 0.2 x 64 g 

3) 3 x 108 g 

4) 2 x 16 g 

5) (2x1 + 16) g 

6) (23 + 35.5) g 

7) 4 g 

8) 3 g 
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Mole Concept 



Solutions to Exercise 3 



1) 1/56 x 6.022 x 10 23 

2) 1/64 x 6.022 x 10 23 

3) 1/108 x 6.022 x 10 23 

4) 1/(2 x 16) x 6.022 x 10 23 

5) 1/(1 + 1 + 16) x 6.022 x 10 23 

6) 1/(23 + 35.5) x 6.022 x 10 23 

7) 1/4 x 6.022 x 10 23 

8) 1/3 x 6.022 x 10 23 
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Some exercises 



Exercise 4 

Suppose a biased coin is tossed 2 times. The probability 
distribution for 0, 1 and 2 heads are given by 0.36, 0.48 and 
0.16, respectively. Find the mean and standard deviation of the 
number of heads. 

Answer 

Let x be the number of heads. The distribution given is 

x P{x) 

0 0.36 

1 0.48 

2 0.16 
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Microstates and Macrostates 
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Some exercises 



The mean is n = £ xP(x) = 0 x 0.36 + 1 x 0.48 + 2x0.16 = 0.8. 

(Out of 2 tosses, we only get 0.8 heads on average. Obviously, 
this coin is biased towards the tail.) 

X X — fj, (x — /^) 2 

0 -0.8 0.64 

1 0.2 0.04 

2 1.2 1.44 

The standard deviation is 

^ = ^E(x-fi) 2 P(x) 

= V0.64 x 0.36 + 0.04 x 0.48 + 1.44 x 0.16 = 0.48. 
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Consider a particle with 2 energy levels: e\ and e 2 . Suppose we 
have N = 2 particles. And there are n\ particles at energy level 
ei, and at e^. 

The following lists all the possible arrangements: 
Macrostates Microstates 

(2, 0) [ ei)Cl ] 
(0, 2) [e 2l e 2 ] 
(1. 1) [ei,£2]. [£2,£l] 

For example, [ei,£2] would mean that particle 1 is at level e\ 
and particle 2 is at level e 2 - This is one particular arrangement. 

A particular arrangement of particles among energy levels is 
called a microstate. 
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(ni,n 2 ) specifying the number of atoms at each level. 

The set of numbers specifying the number of atoms at each 
level is called the macrostate. 

E.g. Consider a particle with 3 energy levels: ey, e 2 and e 3 . 
Suppose there are N = 2 particles. And there are n\ = 2 at 
energy level e\, n 2 = 0 at e 2 , ancl n 3 = 0 at e 3 . 

The macrostate in this case is (n 1 ,n 2 ,n 3 ) = (2, 0, 0). 

Lets write down a useful relation: 

N = n-y + n 2 + n 3 

The total number of particles is equal to the sum of the 
particles at each level. 
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Particles with more than two energy levels 



For more energy levels, the formula generalises to: 



ni!n 2 !n 3 !...rij!... 

or 



The total number of particles is N. The macrostate is specified 
by the number of particles at each energy level: 
(«■!, n 2 , nj, ...). So the total number must the the sum of the 
particles at each level: 

i 

We consider an isolated system that is very well insulated from 
the surrounding. Then the total energy U of the particles 
would remain constant. 
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Particles with more than two energy levels 



In general, a system can have many energy levels. We want to 
know the most likely macrostate (m, n 2 , n 3 , ...) of a system, 
because this would allows us to find the total energy, heat 
capacity, etc. 

The most likely macrostate is the one with the largest number 
of microstates - or arrangements. 

For two energy levels, the number of microstates for the 
macrostate (ni,n 2 ) is 



ni!n 2 ! 

It the number of ways of putting N different objects into 2 
boxes. 
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Particles with more than two energy levels 



There are n-y particles at energy level e\, so the total energy of 
these particles is n\e\. Adding up the energy of the particles at 
all energy levels, we have the energy of the whole system: 

U = n i e i 
i 

fi is a function of the variables (ni, n 2 , ...). 

There are 2 conditions, or constraints, on these variables: 

(i) the total number of particles is a constant, and 

(ii) the total energy is also a constant. 

In order to maximise Q, we can use the method of Lagrange 
multipliers. 
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To find the most probable macrostate 



The procedure is as follows: 

1. We maximise Infi instead of Q. The result would be the 
same because when Inn is a maximum, so is Q. 

2. The reason for using Infi is that it can be simplified for very 
large N using the Stirling's theorem. 

3. Write down the Lagrange function In f2 + X%N + X 2 U. 

4. a and p are new parameters called Lagrange multipliers. 

5. Differentiate this with respect to every n,. 

6. Set the derivative to zero and solve for n i: Xi and A 2 . 

7. This gives the macrostate (ni, 712, -•)■ 
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The algebra 

We need to maximise 

lnn + AiJV + X 2 U 
In order to do so, we need to differentiate this. Start with 

N = m + n 2 + ... + rij + ... 
Differentiating with respect to m gives 

dN _ 1 

drii 

Next, we differentiate 

U = n\e\ + n 2 e 2 + ••• + + ■•• 

This gives 

dU _ 
diii 

Next we need to differentiate Infi. 
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Stirling's Theorem 



The Stirling's Theorem is used for large factorials. 

For real objects, N may be the number of atoms. This could 
be as big as 10 30 , and it takes a long time to calculate. 

Fortunately, there is a simple, approximate formula that is very 
accurate for large numbers: 

In AT! « N\nN-N 
This formula is the Stirling's Theorem. 
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The algebra 

Start with the formula 

n = ^ 

ni!ri2!...nj!... 

Then take the logarithm, 

InQ = In N\ — Inni! — lnn 2 ! — ... — Inn,;! — ... 
and apply Stirling's theorem: 

In Q = N In N— N— (ni In n\— ni) — (n 2 In n 2 — n 2 ) — (n. In n^— n,;) — ... 
Differentiating, we get 

<9lnf2 

= — In rij 

dn-i 

Note that in this differentiation, N in <T2. is to be treated as a 
constant. This is unlike in X\N where it is a function of rij. 
The reason is because f2 is up to us to define, and it is 
sufficient to vary nj. 
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The algebra 



"hese are what we have obtained 

dN 

drii 
dU 

drii 



drii 



- In m 



To maximise the Lagrange function In Q + + A 2 [/, we need 

to differentiate this and set the derivative to zero: 

d\nn , dN , dU 

h Ai h A 2 = 0 

orii orii orii 

Substituting the above results, we get: 

In rii + Xi + A 2 £i = 0 
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Boltzmann's postulate 

Next, we need to find A 2 . In principle, we can use the next 
constraint 

Substituting the solution 

rn = Aexp(A 2 e i ) 

we get 

U = Y, Ae i exp(A 2ei ) 

Then solve for A 2 . 

Unfortunately, this equation is very difficult to solve. Instead, 
we need to take another approach. That is to make use of our 
knowledge of thermodynamics. 
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The most probable macrostate 



We have obtained 

In rij + \i + A 2 £j = 0 
Soving for rij gives the solution: 

rij = Aexp(A 2 e i ) 

where A = exp(Ai). 

Next, we need to determine Ai, or A, and A 2 . 
A can be obtained using 

the condition that the total number of particles is N. This is 
called the normalisation condition. Substituting, we get 

N = A^TexpCA^)- 

Rearranging this gives 

A= N 

Eexp(A 2 £i) 
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Boltzmann Postulate 
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Boltzmann's postulate 



The purpose of doing all these calculations is to find a way to 
calculate thermodynamic quantities like entropy and heat from 
first principles. 

Instead of solving for A 2 directly, we can try and find it in terms 
of known quantities in thermodynamics. We shall start by 
stating the result: 

1 



where k B is the Boltzmann constant and T is the temperature. 
There are different ways to determine this. We shall look at 
the following approach. 
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Boltzmann's postulate 

Since TV is constant, dN = 0. So 

d(ln Q) + X x dN + X 2 dU = 0 

becomes 

dU = — d(lnQ) 
A 2 

Compare with 

dU = TdS. 
Boltzmann postulated that 

S = k B \nn 

where k B is a constant. Substituting this into the above, we 
find that 

1 

A 2 = 

k B T 

This completes the solution for the macrostate (rai, n 2 , ...): 
nj = A exp(A 2 £j) = A exp [ — £ 



k R T 
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Boltzmann's postulate 



To relate A 2 to the entropy formula 

dQ = dU = TdS, 
we look at the Lagrange function In Q + AiJV + A 2 [/ again. 

Previously, we have taken the derivative to find the set of 
numbers (n 1; n 2 ,...) that would maximise the function. We 
have found the solution for n it except for the unknown A 2 . 

For small variations about this solution (n\,ri2, ■■■), the change 
in the Lagrange function would be zero, since it is at a 
maximum. So 

d(ln n) + X t dN + X 2 dU = 0 

Suppose we impose the condition that the total N is fixed, but 
we give some heat to the system so that U increases a little. 
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Boltzmann's postulate 

We shall see later, in the lectures on ideal gas, that k is the 
Boltzmann constant 

k B = — = 1.3807 x 10" 23 J K _1 , 

N A 

where 7? is the ideal gas constant and N A is Avogadro's 
constant. 

The solution for the macrostate tells us how the number of 
particles are distributed in different energy levels: 



rii = A exp 



k B T 

It is called the Boltzmann distribution. 
Historically, the formula 

S = fc B lnQ. 
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Boltzmann's postulate 



was first postulated by Boltzmann. It is so important that it is 
engraved on his tombstone ... 





Example 1 

Minimise the function f(x, y) = x 2 + y 2 , subject to the 
constraint x + y — 2 = 0. 

(i) Do it mentally, or by inspection of the graphs. 

(ii) Do it using Lagrange multiplier. 
Do the results agree? 
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Worked Examples 
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Mentally 



Answer 




1. The contours of f(x,y) are just circles, e.g. x 2 + y 2 = 1, 

x 2 + y 2 = 2, ... The circles get larger as the value increases. 

2. The constraint x + y — 2 = 0 is a straight line with 
intercepts x = 2 or y = 2. 
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Answer: Mentally 




Example 2 

Use your calculator to work out In 10!. Compare your answer 
with the simple version of Stirling's theorem (N In N — N). How 
big must N be for the simple version of Stirling's theorem to be 
correct to within (ii) 5% (ii) 1% ? 

Answer 

ln(10!) = 15.11 whereas 10ln(10)-10 = 13.03. There is less 
than 5% difference for N = 24 and less than 1% for B = 91. 
Stirling's approximation is quite accurate even for relatively 
small N. 
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Using Lagrange multiplier 



1. Let g{x, y) = x + y — 2. The constraint is then g(x, y) = 0. 

2. The Lagrange function is 

f{x, y) + \g(x, y)=x 2 + y 2 + X(x + y~ 2) (1) 

3. The gradient gives: 

1. Differentiate wrt x: 2a; + A = 0. 

2. Differentiate wrt y. 2y + \ = 0. 

4. Solving with the constraint equation gives A = —2 and 
x = y = 1, as before. 
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Boltzmann Distribution 



Example 3 

A system of particles is at a temperature of 2 K. Find the 
Boltzmann factor for each of these energy levels: 
(k B = 1.38 x 10" 23 ) 



1) 1 


x 


10 


-23 


J 


2) 2 


X 


10 


-23 


J 


3) 3 


X 


10 


-23 


J 


4) 4 


X 


10 


-23 


J 


5) 5 


X 


10 


-23 


J 


6) 6 


X 


10 


-23 


J 


7) 7 


X 


10 


-23 


J 


8) 8 


X 


10 


-23 


J 
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Boltzmann Distribution 



Solutions to Example 3 



1) 0.6961 

2) 0.4845 

3) 0.3372 

4) 0.2347 

5) 0.1634 

6) 0.1137 

7) 0.07916 

8) 0.05510 
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Boltzmann Distribution 



Solutions to Example 4 



1) 6.736 x 10 

2) 5.896 x 10 

3) 5.601 x 10 

4) 5.452 x 10 

5) 5.362 x 10 

6) 5.302 x 10 

7) 5.259 x 10 

8) 5.226 x 10 
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Boltzmann Distribution 



Example 4 

There are 10 23 particles. The system has two energy levels 
only: 0 J and 10~ 23 J. Find the number of particles at 0 J 
when the temperature is: 



1) 


1 


K 


2) 


2 


K 


3) 


3 


K 


4) 


4 


K 


5) 


5 


K 


6) 


6 


K 


7) 


7 


K 


8) 


8 


K 



49 



Example 5 

Consider a 10000 distinguishable particles at room temperature, 
298 K. Suppose that each particle has 2 energy levels, 0.01 eV 
and 0.02 eV. Find the number of the particles in each energy 
level. ( Boltzmann's constant is 1.3807 x 10~ 23 J K -1 .) 
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Example 5 



Answer 

Since the number is quite large, we assume that the probability 
at each energy level is given by Boltzmann's distribution 

Aexp(-e/k B T). 

At 0.01 eV, Aexp(-e/k B T) = 0.6778A 
At 0.21 eV, Aexp(-e/k B T) = 0.4594A 
The total is 0.6778A + 0.4594A = 10000. So A = 8794. 

Therefore the numbers are: 

at O.OleV, 0.6778 x 8794 = 5961; 
at 0.02eV, 0.4594 x 8794 = 4040. 
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Example 7 

Consider a large number of distinguishable particles, at 
temperature T.. Each particle has 4 energy levels 0, k B T, 1k B T 
and 3k B T. Calculate the fraction of particles in each energy 
level. 

Answer 

Since the number is quite large, we assume that the probability 
at each energy level is given by Boltzmann's distribution 

Aexp(-e/k B T). 
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Example 6 



Calculate the quantity k B T at room temperature, 298 K. Give 
the answer in eV. ( Boltzmann's constant is 1.3807 x 10~ 23 J 
K _1 . Electron charge is 1.6 x 10~ 19 C.) 

Answer 

In Joules, k B T = 1.3807 x 10 -23 x 298 = 4.114 x 10 -21 J. 

In eV, k B T/e = 4.114 x 10" 21 /l-6 x 10" 19 = 0.026 eV. 

Note: Since 1/40 = 0.025 is quite close to this answer, k B T at 
room temperature is often quoted as 1/40 eV. 
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First, calculate the following: 

at energy 0, exp(— e/k B T) = exp(0) = 1. 
at energy k B T, exp{-e/k B T) = exp(-l) = 0.3679. 
at energy 2k B T, exp(-e/k B T) = exp(-2) = 0.1353. 
at energy 3k B T, exp(-e/k B T) = exp(-3) = 0.0498. 

The sum is 1.553. The fractions are, therefore: 

at energy 0, 1/1.553 = 0.6439. 
at energy k B T, 0.3679/1.553 = 0.2369. 
at energy 2k B T, 0.1353/1.553 = 0.0871. 
at energy 3k B T, 0.0498/1.553 = 0.0321. 

Notice that fraction drops to nearly 1/10 after just 2k B T. 

In statistical mechanics, we often want to know if an energy 
level is likely to be populated at some temperature. If the level 
is much higher than k B T, then it is unlikely to be occupied. 
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Worked example 



Example 8 

There is one mole of atoms at a temperature of 1 K. Each 
atom has energy levels — 10~ 23 J and 10~ 23 J. Find the number 
of atoms at each level. 

Solution 

The formula is Boltzmann distribution 

Hi = Aexp(-si/k B T). 

Given 

e\ = -10" 23 J, e 2 = 1CT 23 J, and T = 1 K. 

We also know that + n 2 = N A (Avogadro constant). 

We need to find the numbers n\ and n 2 . But what is A? 
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Worked example 

Example 9 

Following from the previous example, find the total energy of 
the atoms. 

Solution 

There are n\ atoms with energy e\, so the energy of these 
atoms is nisi 

There are n 2 atoms with energy £ 2 . so the energy of these 
atoms is n 2 e 2 . 

Therefore the total energy is 

U = niei + n 2 £ 2 . 
Using the values from the previous example, we get 

U = (4.876 x 10 23 )(-10" 23 ) + (1.145 x 10 23 )(10" 23 ) = 6.021J. 
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Lets write out all equations: 

n 1 +n 2 = N A 
n x = Aexp(-e 1 /k B T) 

n 2 = ^exp(-£ 2 /fc B T) 

There are three unknowns: n\, n 2 and A. Divide the last two 
equations: 

Til 

-i = exp((£ 2 - e 1 )/k B T). 
n 2 

Let this above fraction be r. So the total is N A is divided into 
the ratio r : 1. Therefore 

n x = x N A = 4.876 x 10 23 

r + 1 

n 2 = x N A = 1.145 x 10 23 

r + 1 
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Worked example 



Example 10 

Following from the previous example, find the probability that 
an atom is at level 1, and the probability that it is at level 2. 

Solution 

The total number of atoms is N A . 

There are ni atoms at level 1 and n 2 atoms at level 2. 

An atom can change level with time. So any one time, 
probability that it is at level 1 is 

number of possible atoms at level 1 ni 
total number of atoms N A 
Probability that it is at level 2 is 

number of possible atoms at level 2 _ n 2 
total number of atoms N A 



4.876 x 10 23 
6.022 x 10 23 

1.145 x 10 23 
6.022 x 10 23 
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Worked example 



Example 11 

There are a number of atoms at a temperature of 1 K. Each 
atom has energy levels — 10~ 23 J and 10~ 23 J. Find the 
probability that an atom is at each level. 

Solution 

This time, we are not given the total number of particles. We 
still have the Boltzmann distribution: 

ni = Aexp{-e\/k B T) 

n 2 = Aexp(-e 2 /k B T) 

but we cannot find A if we don't know the the total number of 
particles. 
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Worked example 



But we know that the probability that an atom is at level 1 is 
number of possible atoms at level 1 n± 

total number of atoms ni+n 2 

If we substitute the Boltzmann distribution, A cancels out! 

ni Xexp(-£!/fc B r) 

ni +n 2 ~~ Xexp(-£i/fe B T) + Aexp(~e 2 /k B T) 

We do know the energy levels and temperature, so we can not 
calculate this. In the same way, the probability that an atom is 
at level 2 is 

n 2 Aexp(-e 2 /k B T) 

ni + n 2 Xexp(-e!/fc B T) +Aexp(-e 2 /k B T) 

The answers are 4.876/6.022 and 1.145/6.022 respectively. 
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